Coagulation test understanding and ordering by medical trainees: Novel teaching approach

Abstract Background Coagulation testing provides a prime opportunity to make an impact on the reduction of unnecessary laboratory test ordering, as there are clear indications for testing. Despite the prothrombin time/international normalized ratio and activated partial thromboplastin time being validated for specific clinical indications, they are frequently ordered as screening tests and often ordered together, suggesting a gap in understanding of coagulation. Methods Based on a needs assessment, we developed an online educational module on coagulation for trainees, incorporating education on testing cost, specificity, and sensitivity. Fifty participating resident physicians and medical students completed a validated premodule quiz, postmodule quiz after completion of the module, and a latent quiz 3 to 6 months after to assess longer‐term knowledge retention. Trainees provided responses regarding their subjective laboratory test‐ordering practices before and after module completion. Results The median premodule quiz score was 67% (n = 50; range, 24%‐86%) with an increase of 24% to a median postmodule quiz score of 91% (n = 50; range, 64%‐100%). There was evidence of sustained knowledge acquisition with a latent quiz median score of 89% (n = 40; range, 67%–100%). Trainees were more likely to consider the sensitivity, specificity, and cost of laboratory investigations before ordering them following completion of the educational module. Conclusions Using the expertise of medical educators and incorporating trainee feedback, we employed a novel approach to the teaching of coagulation to maximize its approachability and clinical relevance. We found sustained knowledge retention regarding coagulation and appropriate coagulation test ordering, and a subjective change to trainee ordering habits following participation in our educational intervention.


Essentials
• Inappropriate international normalized ratio (INR)/activated partial thromboplastin time (aPTT) testing in unselected patients remains a problem in the inpatient setting.
• Medical trainees carry out a large proportion of laboratory test ordering in the hospital.
• Our e-module on INR/aPTT testing enhanced trainee knowledge of coagulation and appropriate testing.
• Innovative and remotely accessible e-learning resources can bridge the coagulation knowledge gap.

| INTRODUC TI ON
Choosing Wisely, an initiative launched by the American Board of Internal Medicine Foundation with the goal of reducing unnecessary laboratory investigations and treatments, has led to greater emphasis on resource stewardship in medicine. However, overutilization of laboratory tests continues to be a problem that results in unnecessary costs, a cascade of additional investigations or treatments, and may even result in patient harm. [1][2][3][4][5] Coagulation testing is a prime opportunity for utilization initiatives, as there are clear indications for testing and harms associated with inappropriate use of these tests. The prothrombin time (PT)/ international normalized ratio (INR) was validated for warfarin monitoring in steady state, while the activated partial thromboplastin time (aPTT) was subsequently validated for heparin monitoring and for screening for hemophilia in affected families. 6 Despite these tests being developed and validated for specific clinical indications, they are frequently ordered as routine screening tests in unselected patients, and often ordered together, which suggests a gap in physician understanding of coagulation, coagulation test limitations, and appropriate test usage. [6][7][8] The excessive ordering of PT/aPTT testing is a particular concern, as the attempt to correct mild elevations of PT/aPTT could result in inappropriate transfusion of frozen plasma (FP). 9,10 FP is often transfused in patients with mild abnormalities in coagulation tests and corrects these values in only a minority of patients. 9 and increased costs to the health care system. 10,11 In addition to the specific harms of unnecessary coagulation testing, iatrogenic anemia secondary to frequent blood draws in the hospital is associated with increased risk of red blood cell transfusion. 4,5 Medical trainees (resident physicians and medical students) conduct a large proportion of laboratory test ordering at academic centers. Previous studies have demonstrated increased laboratory testing in teaching hospitals, compared to nonteaching hospitals, which may be associated with exposure to the training environment. [15][16][17][18] Despite the efforts of Choosing Wisely, in academic hospital medicine there is a culture that values overtesting, with a hidden curriculum that increased ordering of investigations demonstrates medical knowledge and thoroughness. 19,20 Increased laboratory test ordering among resident physicians has been linked to decreased knowledge regarding laboratory test appropriateness, test costs, and other harms of testing. [19][20][21][22] Trainees have also connected the ordering of laboratory tests to their own insecurities and fear of criticism from supervisors for not ordering laboratory tests. 23 A lack of prioritization of teaching on appropriate use of testing and resource stewardship has previously been identified by trainees. 21,23 Coagulation has notoriously been a topic that medical trainees find challenging to learn, and the lack of understanding around coagulation and testing has led to widespread inappropriate ordering and misinterpretation of these tests. 7,8 We developed an online educational module on coagulation tests for trainees, with the goal of enhancing knowledge of appropriate inpatient coagulation laboratory test-ordering practices.

| Root cause analysis and educational module development
We previously conducted a qualitative root cause analysis using a survey and focus group interviews with 10 internal medicine To evaluate the educational impact of the module, a 28-question multiple choice pre-and postmodule quiz assessing knowledge of coagulation and appropriate use of coagulation testing was created and reviewed by hematology faculty with expertise in coagulation (Appendix S1). The quiz and the module underwent a validation process with regard to content, internal structure, response process, and relationship to other variables including level of training, with pilot testing and revisions. 24,25 The quiz had appropriate variability according to trainee level (including hematology and hemostasis fellows who participated in the validation process), and strong interitem reliability. The quiz was hosted on a web-based platform (Survey Monkey: www.survey monkey.com).
Using Articulate 360 software 26 the educational module was created and made available on a mobile-friendly website platform at www.coagt esting.com. The content included in the educational module was based on the needs assessment at our center and included the basics of hemostasis, a simplified coagulation "cascade," common inherited bleeding disorders, disseminated intravascular coagulation, and coagulopathy of liver disease. The educational module was tailored for trainees on a general internal medicine clinical rotation and reviewed the appropriate indications to order a PT/ INR, aPTT, or both in this clinical setting, as well as the differential diagnosis for abnormalities in this testing.

| Data collection
The study participants included resident physicians and medical students on a general internal medicine clinical rotation at our center.
Participating trainees performed a premodule knowledge quiz on coagulation, and a subsequent postmodule knowledge quiz after completing the education module. To assess longer-term knowledge retention, trainees were asked to repeat the knowledge quiz 3 to 6 months following their initial completion of the educational module. Trainees also provided responses regarding their subjective laboratory test-ordering practices before and after completion of the module, using a 5-point Likert scale.
The primary outcome of the study was to evaluate whether the educational module increases trainee knowledge on coagulation and appropriate coagulation test usage, based on postmodule quiz scores.
Secondary outcomes were to evaluate whether there was evidence of sustained knowledge retention on 3-to 6-month postintervention quiz scores (latent quiz) and to determine whether the educational module resulted in a subjective change in trainee testordering practices.
Mean and median quiz scores were collected, with participants' responses described using a Wilcoxon signed-rank test. A P value of<.05 was deemed to be statistically significant.
In the premodule quiz, only 6% of trainees correctly estimated the sensitivity and specificity of the PT and aPTT in detecting a bleeding disorder, with 94% of trainees overestimating the sensitivity and specificity of the aPTT in this setting. In the premodule quiz, only 32% of trainees correctly answered the question regarding the appropriate use of an inpatient aPTT test, and 48% of trainees correctly identified the cost of a PT test (Table 1).

Following completion of the module, 78% of trainees correctly
identified the sensitivity and specificity of aPTT in detecting a bleeding disorder. In the postmodule quiz, 94% correctly answered the question regarding the appropriate use of an aPTT test in the inpatient setting, and 90% correctly identified the cost of a PT test (Table 1).

| Subjective ordering practices
When surveyed about their ordering practices in the premodule quiz, 86% of trainees answered "Agree" or "Strongly Agree" on a 5-point Likert scale when asked whether they are more likely to order a laboratory test if it is listed in a predesigned order set. Forty percent of trainees answered "Agree" or "Strongly Agree" when asked if they would feel uncomfortable not ordering a laboratory test if it is included in a standard predesigned order set.
Of the trainees that participated in the latent quiz, prior to completion of the module, only 32.5% selected "Agree" or "Strongly Agree" on a 5-point Likert scale when asked whether they feel comfortable with their use of coagulation testing in the hospital and feel that they order these tests appropriately. Following completion of the module, 82.5% of trainees answered "Agree" or "Strongly Agree" when asked about their comfort with their appropriate use of coagulation testing. Trainees also expressed that the educational module had a positive impact on their learning. In addition, trainees expressed that they were more likely to consider the sensitivity, specificity, and cost of laboratory investigations before ordering them following completion of the educational module.

| DISCUSS ION
We demonstrated a significant increase in trainee coagulation knowledge after implementation of a targeted educational intervention. The median premodule quiz score was 67%, which increased to a median of 91% postmodule quiz score. In our study, we were also able to demonstrate sustained knowledge retention up to 6 months following completion of the educational module (latent quiz median score of 89%). Trainees also reported a subjective change to their test-ordering practices following completion of our module, and that they would now be more likely to consider sensitivity, specificity, and cost before ordering a laboratory test.
While the PT and aPTT are often considered "routine" tests,  these tests for detecting a bleeding disorder is only 1% to 2%. 7,27 There are many harms associated with inappropriate coagulation testing, including increased risk of exposure to blood components and products, a cascade of additional investigations and referrals, delays to patient care, patient and physician anxiety, iatrogenic anemia, and additional, often unnecessary costs to the health care system. [3][4][5]7,[9][10][11] In a previous study conducted at our center in the emergency department, the reduction of routine use of PT/aPTT testing in unselected patients was associated with an estimated cost saving of US$56,000 per year. 28 As medical trainees order a significant proportion of laboratory tests at academic centers, this is an important group to target in clinical quality improvement interventions. 15,16 Prior work has also highlighted the importance of involving medical trainees in quality improvement initiatives targeting ordering practices. 29 Future directions will include targeted development of the e-module based on trainee feedback to tailor the contents to the specific needs of medical and surgical specialties. For example, our immediate next step will be to create three additional modules: one on coagulation testing for the emergency setting targeting emergency medicine and intensive care trainees, one on coagulation testing in the perioperative setting for surgical and anesthesia trainees, and one on appropriate testing for the ambulatory setting for family medicine trainees. Given the unique learning objectives and clinical contexts of each of these specialties, these tailored modules will feature specialty-specific interactive cases to maintain engagement and clinical relevance.

| CON CLUS IONS
We have developed an online educational module for trainees on inpatient coagulation testing. We have demonstrated sustained knowledge retention regarding coagulation and appropriate coagulation test ordering, as well as a subjective change to trainee ordering habits following participation in our educational intervention. Furthermore, our educational intervention has been accessed by users across several countries. The extent to which individuals have been accessing our educational module internationally suggests to us that there is a need for such a web-based, accessible resource to bridge the coagulation knowledge gap. In the context of the COVID-19 pandemic, which has disrupted traditional medical